In order to prepare an augmented reality space with respect to a real space that is not pre-modeled, a user has to obtain image feature-camera position information about the real space by using a camera, generate local reference coordinates (or matching coordinates), and then match coordinates of a virtual space based thereon. However, since the matching coordinates are generated in an arbitrary location, the user has to manually calibrate a position of the matching coordinates.
Also, through a calibration process of accurately matching a scale between a real space and an augmented reality space, a modeled 3-dimensional (3D) virtual object may be accurately augmented in the augmented reality space in units of the real space, for example, in units of meters (m).
As examples of an existing matching method, a global positioning system (GPS)/compass sensor-based matching method has very low matching precision due to an error in sensor information, and a 2-dimensional (2D) object-based matching method requires a pre-learned image and is not suitable to arbitrary 3D space matching because a matching target is restricted to a 2D plane having a simple shape.
In 3D space-based matching, a user needs to manually calibrate a coordinates position because matching coordinates for augmentation is generated at an arbitrary location. In order to perform such calibration, the user needs expert knowledge related to computer vision/graphics, and if the user performs an inaccurate input, a matching error may be generated due to the inaccurate input.
Also, as an example of a general augmented reality system, KR 10-0980202 discloses a mobile augmented reality system for interaction with 3D virtual objects and a method thereof. The mobile augmented reality system includes a camera attached to a terminal, an image processor generating a 3D virtual object on a hand by using the camera of the terminal, a display unit outputting an image of the 3D virtual object and an image of the hand, and an interaction unit controlling the 3D virtual object correspondingly to movement of the hand, thereby enabling a user to access 3D virtual content anywhere at any time by using a mobile device. As such, KR 10-0980202 is related to a technology for accessing 3D virtual content by using a mobile device, and does not disclose details about automatically generating and calibrating matching coordinates for matching a virtual space.
Accordingly, there is an increasing necessity for a method of automatically generating and calibrating matching coordinates for matching a virtual space.
Meanwhile, nowadays, studies of an interaction technology with tangible content, in which a digital technology, and culture and arts are converged, are receiving a lot of attention. In particular, according to development of augmented reality technologies based on computer graphics and computer vision technologies, there have been attempts to combine virtual digital content to a real world. Also, according to weight lightening and miniaturization of a camera and a head-mounted display (HMD), a wearable computing technology is accelerating. From among various user interface (UI) technologies that are being currently studied, a hand is attracting attention as a natural technology for a wearable computing technology.
There are various general interface technologies for obtaining digital information with respect to an object, a space, and a situation, which interest a user. Examples of a device for such an interface include desktop-based interfaces, such as a mouse, a keyboard, and a remote controller. Such interface technologies may be used to control digital technologies shown on a 2D screen. However, since the interface technologies aim for 2D display, the interface technologies are limited in terms of spaces.
A real space we are living in is a 3D space. When an existing interface for 2D display is used in the real space, a dimension of a space is decreased by 1 and thus the existing interface is restricted.
Accordingly, a 3D interface technology is required to process virtual digital content combined to a 3D space.
Unlike an existing display in a desktop environment, a camera-attached HMD provides a first person environment to a user.
However, in such a camera environment, studies of estimating a finger posture of a bare hand have following problems.
First, a hand is an object having 26 high-dimensional parameters (palm: 6 DOF and 5 fingers: 45=20 DOF). A large computation amount is required in order to estimate a posture of a finger having high dimension.
Second, a hand is an object without texture. Thus, an algorithm of detecting/tracking a feature point-based object from color information is unable to be applied to estimate a finger posture. As such, operations of detecting/tracking a hand and estimating a posture of a finger based on a camera have challengeable conditions.
A WearTrack system is a wearable system using an HMD to which a sensor capable of posture estimation is attached, and a segment magnetic tracker. Systems, such as a virtual touch screen system, AR-Memo, and SixthSense perform 2D interaction based on 2D image coordinates. However, since such systems perform 2D-based interaction, the systems are unable to perform interaction in a 3D space.
Tinmith and FingARtips estimate a posture of a hand by attaching an additional marker to a glove. However, since a size of a segment sensor is too large, Tinmith and FingARtips are not suitable for a user in a wearable environment.
A feature point-based approaching method has also been developed. Such a method estimates a movement of a finger by recognizing a pattern through prior learning. Here, an RGB-D camera, such as Kinect, is fixed to look at a hand of a user, and movement of the hand wearing a glove having a certain pattern is estimated. In the feature point-based approaching method, an additional glove is required to recognize a posture of a finger of the user.
A digits system uses a fingertip tracking method suitable to a wearable device. A depth sensor using a time-of-flight (TOF) method is worn on a wrist and is set such that a self-covering phenomenon of a finger is not generated. According to the digits system, fingers are simply distinguished from each other by using a carving technique, and postures of the fingers are estimated by using relationships between finger joints. However, a sensor needs to be attached to an additional body part, such as a wrist, in addition to an HMD.